The art of music compression is based on psychoacoustic phenomena, e.g., the human hearing system and the human brain can not process certain acoustic information of lesser amplitudes in the presence of other information of higher amplitudes. It is well known that the recent rise in music piracy had its impetus on college campuses when students with fast Internet connections discovered that a sound compression technology called MP3 could pack copies of music that could be reproduced almost perfectly into a file small enough to distribute rapidly on line. These students, ripping songs from CDs, compressing them (typically, into approximately one-tenth their original size), and posting the files on the Web, proved that MP3 and other compression technologies could transform an industry and establish an entirely new way to distribute music whether acquired legitimately or not.
Downloading music is the process of copying a file from a site on the Internet to the user""s hard drive. The commonly used term xe2x80x9cstreaming audioxe2x80x9d refers to music that can be listened to as it arrives without necessarily being stored. (Presently, the most popular streaming format is RealAudio (copyright).) Downloaded music and streaming music are differentxe2x80x94streaming music provides instant gratification but sometimes poor playback quality, and it does not necessarily capture the content. Downloading, which involves moving an entire file before any part of it is accessible, offers the potential for better quality over limited bandwidth networks and, fueling piracy and copyright concerns, deposits the content on a hard drive. However, as used herein, the term downloading refers to both. Already there is a new breed of xe2x80x9csolid-state personal stereo,xe2x80x9d such as the Rio (copyright) of Diamond Multimedia (copyright) and the Nomad (copyright) of Creative Labs (copyright), in which downloaded music is stored on RAM.
Artists and music companies would of course like to protect their music. That is extremely difficult. For example, once music is in analog form, which it must be if conventional speakers are used, it is very difficult to prevent unauthorized copying. And there is no practical way to prevent compressed music files from being distributed over the Internet, for example, by way of E-mail.
There is clearly a need for some kind of protection scheme for electronic music distribution (commonly referred to as EMD) that will protect artists and the music companies, at least partially, yet be accepted by the record-buying public and the consumer electronics companies. From a practical standpoint, the goal is not to prevent unauthorized music distribution in any form. The goal of the present invention is to prevent play or processing of pirated music (so that distribution of pirated copies over the Internet, which from a practical standpoint just cannot be prevented, achieves nothing of value). The present invention is also applicable to the distribution of audio/video files. In the appended claims, the term xe2x80x9cmusicxe2x80x9d is to be taken as embracing video and other forms of entertainment programming material as well.
In the long run, any protection scheme must have the cooperation of the consumer electronics manufacturers and it must be based on the rule of law. Thus, if the consumer electronics industry and the personal computer manufacturers agree that their music players and PCs will conform to standards that implement a copy-protection scheme such as that of the present invention, and there are lawsxe2x80x94such as those that exist in the United Statesxe2x80x94that make it a crime to foil a player designed to protect against play of pirated music, then a copy-protection scheme can be effective. The present invention is designed to work with a player for downloaded (or other) music that operates in accordance with certain rules, the assumption being that the player operation will not be illegally modified under penalty of law.
Before considering the invention itself, it will be helpful to understand several different scenarios that can arise in electronic music distribution:
(1) The medium (CD, broadcast, etc,) over which a compressed file is received should not be ignored. If a compressed file is received over a secure channel, i.e., one whose very nature is an assurance that the file was most likely sent with authorization of the music rights holder, it is not unreasonable to assume that the file should be played. An example is that of a record company that sells music in compressed form over the Internet, from a Web site that encrypts the music file. The file can be used only if the recipient has the decryption key, and that usually happens only if a payment has been made for it. Thus there is little need to prevent play of a compressed file that arrives over a secure channel.
(2) Consider next music that is received via a communications medium such as the Internet, over a channel that is not secure, and is received in compressed form. One possibility is that the music was originally released on a CD or DVD and it was the rights holder""s intent that the music not be copied. If it is now being received in compressed form over an insecure channel, it is certainly possible that the music is being received from an unauthorized source which compressed it and is now distributing it, perhaps for a fee and perhaps for free.
(3) On the other hand, there are many artists who freely distribute their music from Web sites in compressed form. Very often this is the only way for a group to make a name for itself. This legitimate compressed file must be distinguishable from the pirated compressed file of the previous case. There must be a way to prevent play in one case and allow it in the other, even when both files are received over an insecure channel.
(4) When music is released on a CD or DVD, it is in the form of linear PCM (pulse code modulation) or, in the case of DVD, it may have been losslessly compressed to pack the data more efficiently. We are assuming that the music is to be played on a player that operates on a bit stream, compressed or linear PCM. Music files, in analog or any digital form, can be compressed into an MP3, AC3 or some other standard lossy compressed format. If the test used by a player to accept or reject a file always approves of a file that has not been subjected to lossy compression, be it analog or digital, then to foil the system it would only be necessary to expand the file after it is first received in compressed form from an illegitimate source. So to be effective, a protection scheme must not approve a file for play or other processing simply because it is not compressed at the point in the chain at which the xe2x80x9cpiracyxe2x80x9d test is applied.
In accordance with the principles of my invention, in one embodiment thereof, two watermarks or tags are inserted into the music by the music publisher. One watermark is robust. By this is meant that it will not be destroyed by compression. (Hereinafter, the term xe2x80x9ccompressionxe2x80x9d excludes compression that is lossless; as used hereinafter, xe2x80x9ccompressionxe2x80x9d results in some information loss.) The other watermark is weakxe2x80x94it is designed to be destroyed by compression. The robust mark tells the player that the music is protected (xe2x80x9cthis music is not authorized to be delivered in compressed form over an insecure channelxe2x80x9d), i.e., if the music is found to have been compressed and it was delivered over an insecure channel, then it should not be played or otherwise processed.
The player includes circuitry (or works in conjunction with circuitry or a data processor) that may not only check the kind of channel over which the music file was received, but also can tell if the music was compressed by checking whether the second watermark, the weak one, is missing. If the weak watermark is missing in the presence of the strong watermark, it is an indication that at some time in the history of the received music file, the music was compressed.
The basic theory is that music that is derived from a compressed source over an insecure channel, and which the rights holder has explicitly marked as unauthorized for such distribution, should not be played. For play to be prevented, the music must have been compressed at some time during its life (even if it is now in linear PCM, i.e., uncompressed, or analog form), and it must have been received over an unknown or insecure channel. Of course, some music publishers want their music to be distributed in compressed form and they want it to be copied. They will omit the robust watermark. The robust watermark is an indication that the music is protected, that there are rules that govern copying. The rules themselves may be included in the watermark if it is complex (e.g., how many copies may be made, etc.), or the watermark may simply represent the simple rule that if the music is received over an insecure channel and it was at some time in its history compressed, then it should not be copied and/or played. If the weak watermark is missing, a sign that at some time in its history the music was compressed, then the music is not played. It can now be shown that this scheme satisfies each of the four cases considered above, as well as others.
(1) The first case is that of a music file in any form received over a secure channel, i.e., one whose very nature is an assurance that the file was most likely sent with authorization of the music publisher. Whether there are no watermarks, or one or two, such music is played.
(2) The next case is that of music whose publisher wants it to be protected, that is received via a communications medium such as the Internet, over a channel that is not secure, and is received in compressed form. The presence of the robust watermark is an indication that it was the publisher""s intent that the music not be copied. If it is now being received in compressed form over an insecure channel without the weak watermark, i.e., the weak watermark was destroyed by compression, the music is not played. (In general, the robust watermark may contain an elaborate set of rules pertaining to use and copying, and the xe2x80x9cno copyingxe2x80x9d example is just thatxe2x80x94an example. The basic point is that the robust watermark in effect says that there are rules, explicit or implicit, that control use and copying. The xe2x80x9cmissingxe2x80x9d weak watermark serves simply to establish that at some time in the past history of this particular piece of music, it was in compressed form.)
(3) On the other hand, those artists who freely distribute their music from Web sites in compressed form need have no fear that their music will not be playable. That music will not be published with the robust watermark. The absence of the robust watermark is a signal that the music can be played with no restrictions.
(4) The system can not be compromised by first decompressing a compressed file. If that is done, the weak watermark will still be missing and the music will not be played. In fact, even if the music is decompressed and converted to analog form, it will still not be played. The absence of the weak watermark (in the presence of the robust watermark) indicates that at some time in the past the music was compressed, and that is the test for whether the rules associated with the robust watermark are to be followed.
The system is not foolproof. There is no doubt, for example, that computer users will compress original CD music, and store and play it on their portable players such as the Diamond (copyright) Rio (copyright). But the object of the invention is to prevent the broadcast or E-mailing of playable music over the Internet. From a practical standpoint, it takes too long to transmit an uncompressed music file by Email, so the transmission of uncompressed music files is not yet a serious threat. And if a protected (with a robust watermark) compressed file is transmitted, it will not be playable on conforming equipment because the weak watermark will have been destroyed during the compression.
Interestingly, much work has been done in developing robust watermarks that will not be destroyed during compression. See, for example, Digital Copyright Protection, by Peter Wayner, AP Professional, 1997. Any of the known watermarks (also called tags) may be used for the robust watermark of the invention. But the present invention also relies on a watermark that is precisely what up to now has been avoidedxe2x80x94a code or signal that is destroyed by compression. (The whole purpose of a watermark is to survive compression so that the source of a pirated file can be determined by inspecting the watermark.) Many of the previously rejected watermarks may be used for this purpose. But it is also easy to construct new, deliberately weak watermarks.
Ideally, the weak watermark should be functionally related to the music being protected in order to prevent xe2x80x9cspoofing.xe2x80x9d For example, consider a watermark that is a standard digital signature (an encrypted hash of some portion of the linear PCM music file) whose bits replace the least significant bits in some sequence of digital samples. (This replacement has almost no effect on the music itself.) At least some of these least significant bits are invariably irretrievably lost during compression. That the digital signature can not be authenticated is a sign that the file was compressed. Of course, any single bit error will also erroneously cause the player to think that the file was compressed (i.e., the weak watermark is missing) because it will not be possible to authenticate it. For this reason, many digital signatures can be repeatedly employed. If only one of 50 is authenticated, for example, it may be taken that there was no previous compression. (In such a case, the weak watermark of the invention can be thought of as one or more of the digital signatures, the weak watermark being destroyed only if all of the digital signatures are destroyed.)
It is of interest to note that the invention works only because standard forms of music compression do not work perfectly. Theoretically, if music is compressed perfectly, then there should be no remnant of anything that is not necessary for the music itself. The art of developing watermarks is to embed in a file a signal that remains in the file in spite of the compression. This signal is the robust watermark of the invention. (To develop a weak watermark that is destroyed by compression is easy.)
In its broadest form the invention does not require use of two separate watermarks. A single watermark may suffice to facilitate the detection of pirated music. The watermark must be such that it is altered by compression of the music in a detectable manner but a remnant of the watermark must still remain after compression. The reason for the double requirement is apparent. Compression must alter the watermark in a way that a player can tell that the music containing the watermark was compressed. Otherwise, there is no way to tell that the music was compressed. However, were compression to eliminate the watermark entirely, there would be no way for the player to tell that the music is to be protected at all. That is why, if only a single watermark is used, some remnant of the watermark must survive compression. Thus in its broadest form, the method of the invention facilitates the detection of pirated music by inserting watermark information in music that is to be protected against piracy, the watermark information withstanding compression to the extent that it remains detectable but is changed in a manner that remnants thereof are recognizable if the music is compressed.
In the absence of any indication that the music was originally protected, i.e., a robust watermark is absent or there is no remnant of any watermark, the player processes the music without any restrictions. Even if originally there was a watermark, the consumer is not penalized if there is none nowxe2x80x94he/she is given the benefit of the doubt over whether the music file was originally protected and the protection was somehow circumvented, or the music file was released by the publisher with no restrictions on copying.